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We study the decay B~ -> DK*~ using a sample of 379xl0 6 T(4S) -> BB events collected with the BABAR 
detector at the PEP-II B- factory. We perform a "GLW" analysis where the D meson decays into either a CP-even 
(CP+) eigenstate {K+K~, tt+tt^), CP-odd (CP-) eigenstate (K^tt , K%fa K°uj) or a non-CP state {K~-k + ). We 

also analyze D meson decays into K + ir~ from a Cabibbo-favored D decay or doubly suppressed D° decay ("ADS" 
analysis). We measure observables that are sensitive to the CKM angle 7: the partial-rate charge asymmetries Acp±, 
the ratios Hcp± of the P-decay branching fractions in CP± and non-CP decay, the ratio IZads of the charge- 
averaged branching fractions, and the charge asymmetry Aads of the ADS decays: Acp+ — 0.09 ± 0.13 ± 0.06, 
A C p- = -0.23±0.21±0.07, K CP+ = 2.17 ± 0.35± 0.09, K CP - = 1.03 ± 0.27± 0.13, K ADS = 0.066 ± 0.031 ± 0.010, 
and Aads — — 0.34± 0.43 ± 0.16, where the first uncertainty is statistical and the second is systematic. Combining 
all the measurements and using a frequentist approach yields the magnitude of the ratio between the Cabibbo- 
suppressed and favored amplitudes, re = 0.31 with a one (two) sigma confidence level interval of [0.24, 0.38] ([0.17, 
0.43]). The value r p = is excluded at the 3.3 sigma level. A similar analysis excludes values of 7 in the intervals 
[0, 7]°, [55, 111]°, and [175, 180]° ([85, 99]°) at the one (two) sigma confidence level. 
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I. INTRODUCTION 

The Standard Model accommodates CP violation 
through a single phase in the Cabibbo-Kobayashi- 
Maskawa (CKM) quark mixing matrix V [1]. The self 
consistency of this mechanism can be tested by over- 
constraining the associated unitarity triangle [2, 3] using 
many different measurements, mostly involving decays 
of B mesons. In this paper we concentrate on the angle 
7 = axg{-V ud V* b /V cd V* b ) by studying B meson decay 
channels where b — ► cus and b — > ucs tree amplitudes in- 
terfere. We use two techniques, one suggested by Gronau 
and London [4] and Gronau and Wyler [5] (GLW) and the 
other suggested by Atwood, Dunietz and Soni [6] (ADS) 
to study 7. Both techniques rely on final states that can 
be reached from both D° and D° decays. As discussed 
in Ref. [7] the combination of the GLW and ADS observ- 
ables can be very useful in resolving certain ambiguities 
inherent in each of the techniques. In this paper we use 
the decay B~ -> DK*~ (892) [8] to measure the GLW 
and ADS observables. 

In the GLW analysis the D meson [9] from B~ — > 
DK*~ decays into cither a CP-cvcn (CP+) eigen- 
state (K + K~, n + TT~) or a CP-odd (CP—) eigenstate 
(K s tt°, K s (j), K s uj). The size of the interference between 
the two competing amplitudes depends on the CKM an- 
gle 7 as well as other parameters that are CP-conserving, 
discussed below. References [4, 5] define several observ- 
ables that depend on measurable quantities: 



Acp± 



T(B- 



D° CP ±K*- 



+ r(B 4 



CP± 



K* 



T(B- 
r(B~ - 



D° CP ±K* 



) + r(B+ 
i - r{B+ - 



D% n K* 



CP± 



K* 



Y(B- 



1J CP±^ 



) + r(B^ 



d° cp± k*+y 



Here D^ p± refers to a neutral D meson decaying into 
either a CP+ or CP— eigenstate. 

1Zcp± and Acp± depend on the physical parameters 
as follows: 



1Zcp± = 1 + r B ± 2t\b con 5b cos 7, 
Acp± = ±2r B sin5 B sin7/7?. ( 7P±- 



(1) 
(2) 



Here Tb is the magnitude of the ratio of the suppressed 
B~ -> D°K*- and favored B~ -> D°K*~ decay ampli- 
tudes, respectively, and 5b is the CP-conserving phase 
difference between these amplitudes. In this analysis we 
neglect the effects of CP violation in D meson decays, 
and, as justified in Ref. [10], the very small effect of D°D° 
mixing. 
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1Zcp± is calculated using: 
N C p± 



Kcp± 



N, 



non — CP 



^non-CP 
£CP± 



(3) 



where Ncp± and N non _cp are the event yields for the 
CP and non-CP modes, respectively and e non -cp and 
ecp± are correction factors that depend on branching 
fractions and reconstruction efficiencies. Acp± is calcu- 
lated using the event yields split by the charge of the B 
meson. 

We define two additional quantities whose experimen- 
tal estimators are normally distributed even when the 
value of tb is comparable to its uncertainty: 

x± = r B cos(6b ± 7) (4) 
R C p+(l T A C p+) - Rcp- (1 T A C p-) 
4 

Since x± are also directly measured in Dalitz-plot analy- 
ses [11], the different results can be compared and com- 
bined with each other. We note that an additional set 
of quantities measured in Dalitz-plot analyses, y± = 
r_Bsin(c)B ± 7), are not directly accessible through the 
GLW analysis. 

In the ADS technique, B~ — ► DK*~ decays to 
[K + tt~]e>K*~ , where [K + ir~]r, indicates that these par- 
ticles are neutral D meson (D° or D ) decay products. 
This final state can be reached from B — > D°K*~ and 
the doubly-Cabibbo-suppressed decay D° — ► K + it~ or 
B~ —> D°K*~ followed by the Cabibbo- favored decay 
D° -> K + tt~. In addition, the final state [K ~Tr + } D K 
is used for normalization. We label the decays where 
the K and K* have the same (opposite) charge as "right 
(wrong) sign" where the labels reflect that one mode oc- 
curs much more often than the other. 

In analogy with the GLW method we define two mea- 
surable quantities, IZads and Aads, as follows: 



TIads 

•A ADS 



r(B~ 



[K+TT-] D K*-)+r(B+ -> [K-n+] D K*+) 



T(B- -> [K -n+] D K*-) + T(B+ -» [K+w-] D K*+) ' 
r(£- -> [K+-k-\ d K*-) - T{B+ -* \K--k+] d K*+) 



T(B- [K+n-] D K*~) +V(B+ -* [K-tt+] d K*+) 

T^ads and A ads are related to physically interesting 
quantities by: 

TIads = r D + r| + 2r D r B cos(5 B + S D ) cosj, (5) 
Aads = 2r D r B sm(5 B + 5 D ) sin j/Kads- (6) 

Here rjj is the magnitude of the ratio of suppressed 
D° — > K + tt~ and favored D° — > K~n + decay am- 
plitudes, respectively, while 5d is the CP-conserving 
strong phase difference between these two amplitudes. 
Both t\d and 8d have been measured and we use the 
values given in Ref. [12]: r D = 0.0578 ± 0.0008 and 
od — 21.9ti2:4 de grees. Estimates for tb are in the range 
0.1 < r B < 0.3 [13, 14]. 

It has been pointed out in Ref. [13] that complications 
due to possible variations in and/or 5b result 
of the finite width of a resonance such as the K* and its 
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overlap with other states can be taken into account using 
an alternate formalism. However, in this paper we choose 
to follow the procedures in Refs. [14, 15] and incorporate 
the effects of the non-K * DKw events and finite width 
of the K* into the systematic uncertainties of our A and 
1Z measurements. 



II. THE BABAR DETECTOR AND DATASET 

The BABAR detector has been described in detail in 
Ref. [16] and therefore will only be briefly discussed here. 
The trajectories of charged tracks are measured with a 
five-layer double-sided silicon vertex tracker (SVT) and a 
40-layer drift chamber (DCH). Both the SVT and DCH 
are located inside a 1.5 T magnetic field. Photons arc 
detected by means of a CsI(Tl) crystal calorimeter also 
located inside the magnet. Charged particle identifica- 
tion is determined from information provided by a ring- 
imaging Cherenkov device (DIRC) in combination with 
ionization measurements (dE/dx) from the tracking de- 
tectors. The BABAR detector's response to various physics 
processes as well as varying beam and environmental con- 
ditions is modeled with simulation software based on the 
Geant4 [17] toolkit. We use EvtGen [18] to model the 
kinematics of B mesons and Jetset [19] to model con- 
tinuum processes (e + e~ — > cc, uu, dd, ss). This analysis 
uses data collected at and near the T(4S) resonance with 
the BABAR detector at the PEP-II storage ring. The data 
set consists of 345 fb _1 collected at the peak of the T(4S) 
(379 xlO 6 BB pairs) and 35 fb" 1 collected 40 MeV be- 
low the resonance peak (off-peak data). 

This analysis is a combined update of the previous 
BABAR ADS' [14] and GLW [15] studies of B~ -> DK*~, 
which used 232 xlO 6 BB pairs. Other new features in 
this analysis include the improvement in background sup- 
pression, the refinement of various candidate selection 
criteria, and an update of the estimation of systematic 
uncertainties. The major change is the choice of neural 
networks in the GLW analysis over Fisher discriminants, 
which were used in the previous analysis. We verify the 
improvements on both signal efficiency and continuum 
background rejection in the GLW decay channels with 
simulated signal and continuum events. The increases in 
signal efficiency range from 3% to 14% for all channels ex- 
cept K®<f>, which has the same efficiency. For continuum 
suppression, the neural networks perform 10% to 57% 
better across all channels except K + K~, which displays 
the same performance. 



III. THE GLW ANALYSIS 

We reconstruct B~ — > DK*~ candidates with the 
subsequent decays K*~ — > K®tt~ , — > n + n~ and 
with the D meson decaying into six decay final states: 
D° -> K-TT+ (non-CP final state); K+K~ , tt+tt- (CP+ 
eigenstates); and K^n , K®(f> 7 K®u) (CP— eigenstates). 



We optimize our event selection criteria by maximizing 
the figure of merit S/ ^/S + B, with S the number of sig- 
nal events and B the number of background events, deter- 
mined for each channel using simulated signal and back- 
ground events. Kaon and pion candidates (except for the 
pions from K® decays) are selected using a likelihood- 
based particle identification algorithm which relies on 
dE/dx information measured in the DCH and the SVT, 
and Cherenkov photons in the DIRC. The efficiencies of 
the selectors are typically above 85% for momenta below 
4 GeV while the kaon and pion misidentification rates are 
at the few percent level for particles in this momentum 
range. 

The candidates are formed from oppositely charged 
tracks assumed to be pions with a reconstructed invari- 
ant mass within 13 MeV/c 2 (four standard deviations) 
of the known K® mass [3] , m K o . All candidates are 
refitted so that their invariant mass equals m K o (mass 
constraint). They are also constrained to emerge from 
a single vertex (vertex constraint). For those retained 
to build a K*~ candidate, we further require that their 
flight direction and length be consistent with a K® com- 
ing from the interaction point. The K® candidate flight 
path and momentum vectors must make an acute an- 
gle and the flight length in the plane transverse to the 
beam direction must exceed its uncertainty by three stan- 
dard deviations. K*~ candidates are formed from a K® 
and a charged particle with a vertex constraint. We se- 
lect K*~ candidates that have an invariant mass within 
75 MeV/c 2 of the known mean value for a K* [3]. Fi- 
nally, since the K*~ in B~ — > DK*~ is longitudinally 
polarized, we require |cos#ir| > 0.35, where Oh is the 
angle in the K*~ rest frame between the daughter pion 
momentum and the parent B momentum. The helicity 
distribution discriminates well between a B-meson decay 
and a false 5-meson candidate from the continuum, since 
the former is distributed as cos 2 Oh and the latter has an 
approximately flat distribution. 

Some decay modes of the D meson contain a neutral 
pion. We combine pairs of photons to form tt° candi- 
dates with a total energy greater than 200 MeV and an 
invariant mass between 115 and 150 MeV/c 2 . A mass 
constrained fit is applied to the selected w° candidate 
momenta. Composite particles ((f) and u>) included in 
the CP— modes are vertex-constrained. Candidate <fi 
(lu) mesons are constructed from K + K~ (7r + 7r~7r°) par- 
ticle combinations with an invariant mass required to 
be within two standard deviations, corresponding to 12 
(20) MeV/c 2 , from the known peak values [3]. Two fur- 
ther requirements are made on the uj candidates. The 
magnitude of the cosine of the helicity angle Oh between 
the D momentum in the rest frame of the u> and the nor- 
mal to the plane containing all three decay pions must 
be greater than 0.35 (this variable has a cos 2 Oh distri- 
bution for signal candidates and is approximately flat for 
background). The Dalitz angle [20] 0d is defined as the 
angle between the momentum of one daughter pion in 
the u> rest frame and the direction of one of the other 
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two pions in the rest frame of the two pions. For sig- 
nal candidates, the cosine of the Dalitz angle follows a 
sin 2 8d distribution, while it is approximately flat for the 
background. Therefore we require the cosine of the Dalitz 
angle of signal candidates to have a magnitude smaller 
than 0.8. 

All D candidates are mass constrained and, with the 
exception of the if!? 71-° final state, vertex constrained. We 
select D candidates with an invariant mass differing from 
the known mass [3] by less than 12 MeV/c 2 for all chan- 
nels except K° s tt° (30 MeV/c 2 ) and K°oj (20 MeV/c 2 ). 
These limits are about twice the corresponding RMS 
mass resolutions. 

Suppression of backgrounds from continuum events is 
achieved by using event-shape and angular variables. The 
B meson candidate is required to have |cos#t| < 0.9, 
where 9t is the angle between the thrust axis of the B 
meson and that of the rest of the event. The distribution 
of | cos 9t I is uniform in BB events and strongly peaked 
near 1 for continuum events. 

A neural network (NN) is used to further reduce the 
e + e~ — > qq (q = u, d, s, c) contribution to our data sam- 
ple. Seven variables are used in the NN with three being 
the angular moments Lo, L\ and L 2 - These moments 
are defined by Lj — P* I cos P where the sum is 
over charged and neutral particles not associated with 
the B-meson candidate. Here p* (9*) is the momen- 
tum (angle) of the ith particle with respect to the thrust 
of the candidate B meson in the center-of-mass (CM) 
frame. Additional details on the moments can be found 
in Ref. [21]. The NN also uses the ratio R 2 = H 2 /H of 
Fox- Wolfram moments [22], the cosine of the angle be- 
tween the B candidate momentum vector and the beam 
axis (cos 9b), cos 9t (defined above), and the cosine of 
the angle between a D daughter momentum vector in 
the D rest frame and the direction of the D in the B 
meson rest frame (cos 9h{D)). The distributions of all 
the above variables show distinct differences between sig- 
nal and continuum events and thus can be exploited by 
a NN to preferentially select BB events. Each decay 
mode has its own unique NN trained with signal and 
continuum Monte Carlo events. After training, the NNs 
are then fed with independent sets of signal and con- 
tinuum Monte Carlo events to produce NN outputs for 
each decay mode. Finally, we verify that the NNs have 
consistent outputs for off-peak data (continuum data col- 
lected below the T(AS)) and qq Monte Carlo events. The 
separations between signal and continuum background 
are shown in Fig. 1. We select candidates with neural 
network output above 0.65 (K + K~), 0.82 (tt+tt"), 0.91 
(K°tt°), 0.56 {K° s <j>), 0.80 (K°u), and 0.73 {K~-k + ). Our 
event selection is optimized to maximize the significance 
of the signal yield, determined using simulated signal and 
background events. 

We identify B candidates using two nearly indepen- 
dent kinematic variables: the beam-energy-substituted 
mass mES = \J (s/2 + p • p B ) 2 /E 2 — p\ and the energy 
difference AE = E* B — y/s/2, where E and p are energy 
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FIG. 1: Neural network (NN) outputs and results of the NN 
verifications of (a) K+K~, (b) tt+tt", (c) K° s -k° , (d) K°(f>, (e) 
Kgui, and (f) K~n + subsamples of the GLW analysis. The 
samples used to produce the output are shown as histograms. 
The signal (Monte Carlo simulation) is the shaded histogram 
peaking near 1; the continuum (Monte Carlo simulation) is 
the histogram peaking near 0. The off-peak data used to 
check the NN are overlaid as data points. 



and momentum. The subscripts and B refer to the 
e + e~-beam system and the B candidate, respectively; s 
is the square of the CM energy and the asterisk labels 
the CM frame. The tties distributions are all described 
by a Gaussian function Q centered at the B mass with 
a resolution (sigma) of 2.50, 2.55, and 2.51 MeV/c 2 for 
the CP+, CP— and non-CP mode, respectively. The 
AE distributions are centered on zero for signal with 
a resolution of 11 to 13 MeV for all channels except 
K®Tr a for which the resolution is asymmetric and is about 
30 MeV. We define a signal region through the require- 
ment \AE\ < 50 (25) MeV for K° s tt° (all other modes). 
A potentially dangerous background for the B~~ — > 
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D(tt + tt~)K*~ (K®it~) channel is the decay mode B~ — > 
-D(i\r°7r + 7r _ )7r _ which contains the same final-state par- 
ticles as the signal but has a branching fraction 600 
times larger. We therefore explicitly veto any selected 
B candidate containing a K®ir + ir~ combination within 
60 McV/c 2 of the D° mass. 

The fraction of events with more than one acceptable 
B candidate depends on the D decay mode and is always 
less than 8%. To select the best B candidate in those 
events where we find more than one acceptable candidate, 
we choose the one with the smallest \ 2 formed from the 
differences of the measured and world average D° and 
K*~ masses divided by the mass spread that includes 
the resolution and, for the K*~ , the natural width: 

X 2 = Xm d „ + Xm k ,_ (7) 
= (M D o — m£P g ) 2 | {M k ,--M™ g Y 

Simulations show that negligible bias is introduced by 
this choice and the correct candidate is picked at least 
86% of the time. 

From the simulation of signal events, the total recon- 
struction efficiencies are: 12.8% and 12.3% for the CP+ 
modes D -» K+K~ and tt+tt-; 5.6%, 8.9%, and 2.4% for 
the CP- modes D -» K° s ir°, K° s (j> and K°lu; 12.8% for 
the non-CP mode D° -> K~-k + . 

To study BB backgrounds we look in sideband regions 
in AE and mo- We define the AE sideband in the in- 
terval 60 < AE < 200 MeV for all modes. This region is 
used to determine the combinatorial background shapes 
in the signal and bid sideband. We choose not to use 
a lower sideband because of the D* K* backgrounds in 
that region. The sideband region in to_d is slightly mode 
dependent with a typical requirement that vhd differs 
from the D° mass by more than four and less than 10 
standard deviations. This region provides sensitivity to 
background sources which mimic signal both in AE and 
toes an d originate from either charmed or charmless B 
meson decays that do not contain a true D meson. As 
many of the possible contributions to this background 
are not well known, we measure its size by including the 
rap, sideband in the fit described below. 

An unbinncd extended maximum likelihood fit to the 
toes distributions of selected B candidates in the range 
5.2 < toes < 5.3 GeV/c 2 is used to determine signal 
and background yields. We use the signal yields to cal- 
culate the CP- violating quantities Acp and IZcp- We 
use the same mean and width of the Gaussian function 
Q to describe the signal shape for all modes considered. 
The combinatorial background in the toes distribution is 
modeled with the so-called "ARGUS" empirical thresh- 
old function A [23]. It is defined as: 

A(m ES ) oc toesVi - x 2 cxp~^~ x2 \ (8) 

where x = toes /E max and P max is the maximum mass for 
pair-produced B mesons given the collider beam energies 



and is fixed in the fit at 5.291 GeV/c 2 . The ARGUS 
shape is governed by one parameter £ that is left free in 
the fit. We fit simultaneously toes distributions of nine 
samples: the non-CP, CP+ and CP— samples for (i) 
the signal region, (it) the rnn sideband and (Hi) the AE 
sideband. In addition the signal region is divided into two 
samples according to the charge of the B candidate. We 
fit three probability density functions (PDF) weighted by 
the unknown event yields. For the AE sideband, we use 
A. For the mp, sideband (sb) we use a s b-A ± b s fQ, where 
Q accounts for fake- 1? candidates. For the signal region 
PDF, we use a ■ A + b ■ G pea k ± c • G signal , where b is scaled 
from b s b with the assumption that the number of fake D 
background events present in the signal region is equal to 
the number measured in the mp, sideband scaled by the 
ratio of the rap, signal- window to sideband widths, and 
c is the number of B^ — > DK*^ signal events. The non- 
CP mode sample, with relatively high statistics, helps 
constrain the PDF shapes for the low statistics CP mode 
distributions. The AE sideband sample helps determine 
the A background shape. In total, the fit determines 19 
event yields as well as the mean and width of the signal 
Gaussian and the ARGUS parameter £. 

Since the values of £ obtained for each data sample 
are consistent with each other, albeit with large statis- 
tical uncertainties, we have constrained £ to have the 
same value for all data samples in the fit. The simula- 
tion shows that the use of the same Gaussian parameters 
for all signal modes introduces only negligible systematic 
corrections. We assume that the fake D backgrounds 
found in the rap sideband have the same final states as 
the signal and we fit these contributions with the same 
Gaussian parameterization. 

The fake D background is assumed to not violate CP 
and is therefore split equally between the B~ and B + 
sub-samples. This assumption is consistent with results 
from our simulations and is considered further when we 
discuss the systematic uncertainties. The fit results are 
shown graphically in Fig. 2 and numerically in Table I. 
Table II records the number of events measured for each 
individual D decay mode. 

TABLE I: Results from the fit. For each GLW D mode, we 
give the number of measured signal events, the fake D contri- 
bution, Acp and 1Zcp- The fake D contribution is calculated 
by scaling the number of fake D events found in the mo 
sideband region to the signal region. The uncertainties are 
statistical only. We also show the number of measured signal 
events split by the B charge for CP+ and CP— modes. 





# Signal 


# Fake D 


Acp 


TIcp 


Non-CP 


231 ± 17 


5.0 






CP+ 


68.6 ± 9.2 


0.3 


0.09 ±0.13 


2.17 ±0.35 


(B + ) 


31.2 ± 6.2 








(B-) 


37.4 ± 6.8 








CP- 


38.5 ± 7.0 


0.0 


-0.23 ±0.21 


1.03 ±0.27 


(B+) 


23.0 ± 4.8 








(B~) 


15.5 ± 5.2 
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FIG. 2: Distributions of jties in the signal region for (a) the 
non-CP modes in B decays, (b) the CP+ modes in B + , 
and (c) B~ decays and (d) the CP— modes in B + and (e) 
B~ decays. The dashed curve indicates the total background 
contributions, which include the fake D backgrounds. 



TABLE II: Number of signal events from the GLW fit for 
individual D decay modes studied in this analysis. We also 
provide the selection efficiencies (in %) and total correction 
factors (e c ). The uncertainties are statistical only. 





# Signal 


Selection Efficiency (%) 


ec (10- 4 ) 


Non-CP 








K~TT + 


231 ± 17 


12.76 ± 0.09 


48.48 ± 0.96 


CP+ 








K + K~ 


41 ± 7 


12.78 ± 0.05 


4.90 ± 0.16 




28 ± 6 


12.34 ± 0.05 


1.72 ± 0.11 


CP- 








K° s n° 


21 ± 7 


5.59 ± 0.03 


4.10 ± 0.50 


KU 


8 ± 3 


8.90 ± 0.04 


1.30 ± 0.11 


K° s lo 


9 ± 4 


2.35 ± 0.02 


1.49 ± 0.30 



Although most systematic uncertainties cancel for 
Acp, a charge asymmetry inherent to the detector or 
data processing may exist. We quote the results from the 
study carried out in Ref. [24] , where we used B~ — ► D°ir~ 
(with D° decays into CP or non-CP eigenstates) events 
from control samples of data and simulation to measure 
the charge asymmetry. An average charge asymmetry of 
A c h = (— 1.6 ± 0.6)% was measured. We add linearly 



the central value and one-standard deviation in the most 
conservative direction to assign a systematic uncertainty 
of 0.022. The second substantial systematic effect is a 
possible CP asymmetry in the fake D background that 
cannot be excluded due to CP violation in charmless B 
decays. If there is an asymmetry *4f a kc d, then the sys- 
tematic uncertainty on Acp is -4f a kc d x b/c, where b is 
the contribution of the fake D background and c the sig- 
nal yield. Assuming conservatively that |-4f a kc d\ < 0.5, 
we obtain systematic uncertainties of ±0.003 and ±0.040 
on Acp+ and Acp- respectively. Note that since we do 
not observe any fake D background in CP— modes, we 
use the statistical uncertainty of the signal yield from the 
fit to estimate this systematic uncertainty. 

Since IZcp is a ratio of rates of processes with different 
final states of the D, we must consider the uncertain- 
ties affecting the selection algorithms for the different D 
channels. This results in small corrections which account 
for the difference between the actual detector response 
and the simulation model. The main effects stem from 
the approximate modeling of the tracking efficiency (a 
correction of 0.4% per pion track coming from a K® and 
0.2% per kaon and pion track coming from other candi- 
dates), the reconstruction efficiency for CP— modes 
of the D° (1.3% per K° s in K° s <j> mode and 2.0% in K° s tt° 
and K®u>), the tt° reconstruction efficiency for the K®tt° 
and K^[tt + tt^tt°] lj channels (3%) and the efficiency and 
misidentification probabilities from the particle identifi- 
cation (2% per track). The corrections are calculated by 
comparing data and Monte Carlo using high-statistics 
and high-purity samples. Charged kaon and pion sam- 
ples obtained from D- meson decays (D* + — > D°tt + ) are 
used for particle identification corrections. For track- 
ing corrections, we use r-pair events where one r de- 
cays to a muon and two neutrinos and the other de- 
cays to fPhv where h is a K or a it. B° — ► (j>K® and 
B° — > tt + D~(D~ — > K®tt~) decays are used for K® cor- 
rections, and 7T° correction factors are calculated using 
t — > pv and t — ► 7zv samples. The total correction fac- 
tors, which also include branching fractions and selec- 
tion efficiencies, used in the calculation of 1Zcp± (Eq- 3) 
are given in Table II. Rcp+ {Rcp-) is calculated us- 
ing the sum of the individual CP+ (CP—) correction 
factors. Altogether, the systematic uncertainties due to 
the corrections equal ±0.078 and ±0.100 for 1Zcp+ and 
IZcp-, respectively. The uncertainties on the measured 
branching fractions [3] and efficiencies for different D de- 
cay modes, are included in the calculation of the system- 
atic errors due to these corrections. 

Another systematic correction applied to the CP- 
measurements arises from a possible CP+ background 
in the K®<f) and K®u> channels. In this case, the observed 
quantities A°Qp_ 



and 7?.pp_ are corrected: 



A C p- = (1 + e)A°S_ - eA C p+; TZ C p- 



7~?obs 
/X -CP- 

(l + e)' 



where e is the ratio of CP+ background to CP— signal. 
An investigation of the D° -> K~ K+ K° s Dalitz plot [25] 



11 



indicates that the dominant background for D° — ► K®<fi 
comes from the decay a (980) — ► K + K~, at the level 
of (25 ± 1)% of the size of the 4>K® signal. We have no 
information for the uiK° s channel and assume (30 ± 30)% 
of CP+ background contamination. The K®tt° mode has 
no CP+ background. The value of e for the combination 
of CP— modes is (11 ± 7)%. The systematic uncertainty 
associated with this effect is ±0.02 and ±0.06 for Acp- 
and TZcp-i respectively. 

To account for the non resonant K®tt~ pairs in the K* 
mass range we study a model that incorporates S-w&ve 
and P-wave pairs in both the b — > cus and b — > ucs am- 
plitudes. The P-wave mass dependence is described by a 
single relativistic Breit-Wigner while the 5-wave compo- 
nent is assumed to be a complex constant. It is expected 
that higher order partial waves will not contribute signifi- 
cantly and therefore they are neglected in the model. We 
also assume that the same relative amount of S and P- 
wave is present in the b — ► cus and b — ► ucs amplitudes. 
The amount of 5-wave present in the favored b — > cus 
amplitude is determined directly from the data by fit- 
ting the angular distribution of the K®n system in the 
K* mass region, accounting for interference [26]. From 
this fit we determine that the number of non- if* Kgir~ 
events is (4 ± 1)% of the measured signal events. To es- 
timate the systematic uncertainties due to this source we 
vary all the strong phases between and 2*7r and calculate 
the maximum deviation between the S'-wave model and 
the expectation if there were no non-resonant contribu- 
tion for both A C p± [Eq. (2)] and TZ C p± [Eq. (1)]. This 
background induces systematic variations of ±0.051 for 
Acp± and ±0.035 for 1Zcp±- 

The last systematic uncertainty is due to the assump- 
tion that the parameters of the Gaussian and ARGUS 
functions are the same throughout the signal region, AE 
and tod sidebands. We estimate the uncertainties by 
varying the width and mean of the Gaussian and £ of the 
ARGUS by their corresponding statistical uncertainties 
obtained from the fit. All the systematic uncertainties 
are listed in Table III. We add them in quadrature and 
quote the final results: 

A C p+ = 0.09 ± 0.13(stat.) ± 0.06(syst.) 
A C p- = -0.23 ± 0.21(stat.) ± 0.07(syst.) 
TZ C p+ = 2.17 ± 0.35(stat.) ± 0.09(syst.) 
K C p- = 1-03 ± 0.27(stat.) ± 0.13(syst.) 

These results can also be expressed in terms of x± 
defined in Eq. (4): 

x+ = 0.21 ± 0.14(stat.) ± 0.05(syst.), 
x_ = 0.40±0.14(stat.)±0.05(syst.), 

where the CP+ pollution systematic effects are included. 
Including these effects increased x+ and x_ by 0.035 ± 
0.024 and 0.023 ± 0.017, respectively. 



TABLE III: Summary of systematic uncertainties for the 
GLW analysis. 



Source 


SAcp+ 


5A C p- 


S1Zcp+ 


SKcp- 


Detection asymmetry 


0.022 


0.022 






Non-resonant Kgir~ bkg. 


0.051 


0.051 


0.035 


0.035 


Same-final-state bkg. 




0.019 




0.061 


Asymmetry in fake D bkg. 


0.003 


0.040 






Efficiency correction 






0.078 


0.100 


Same Q and A shape 


0.003 


0.013 


0.009 


0.025 


Total systematic uncertainty 


0.056 


0.072 


0.086 


0.125 



IV. THE ADS ANALYSIS 

In the ADS analysis we only use D decays with a 
charged kaon and pion in the final state and K *~ de- 
cays to if°7r~ followed by K° s — > tt+tt^. The ADS event 
selection criteria and procedures are nearly identical to 
those used for the GLW analysis. However due to the 
small value of rp the yield of interesting ADS events 
(i.e. B~ [K+n-] D K*- and B+ -> [K--k+] d K*+) 
is expected to be smaller than for the GLW analysis. 
Therefore in order to reduce the background in the ADS 
analysis the Kg invariant mass window is narrowed to 
10 MeV/c 2 and the K*~ invariant mass cut is reduced to 
55 MeV/c 2 . A neural network using the same variables 
as in the GLW analysis is trained on ADS signal and 
continuum MC events and verified using off-peak contin- 
uum data. The separation between signal and continuum 
background is shown in Fig. 3. We select candidates, 
both right and wrong-sign, with neural network output 
above 0.85. All other Kg, K*~, and continuum sup- 
pression criteria are the same as those used in the GLW 
analysis. 

D — > K~n + and K + n~ candidates are used in this 
analysis. Candidates that have an invariant mass within 
18 MeV/c 2 (2.5 standard deviations) of the nominal D° 
mass [3] are kept for further study. We require kaon 
candidates to pass the same particle identification criteria 
as imposed in the GLW analysis. 

We identify P-meson candidates using the beam- 
energy-substituted mass ?7iES and the energy difference 
AE. For this analysis signal candidates must satisfy 
\AE\ < 25 MeV. The efficiency to detect a B~ -» 
D°K*~ signal event where D° — > Kir, after all cri- 
teria arc imposed, is (9.6±0.1)% and is the same for 
D° -» K-n+ and D° -» K+tt~ . In 1.8% of the events 
we find more than one suitable candidate. In such cases 
we choose the candidate with the smallest x 2 defined in 
Eq. (7). Simulations show that negligible bias is intro- 
duced by this choice and the correct candidate is picked 
about 88% of the time. 

We study various potential sources of background us- 
ing a combination of Monte Carlo simulation and data 
events. Two sources of background arc identified in 
large samples of simulated BB events. One source is 
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NN Output 

FIG. 3: Neural network (NN) output and result of the NN ver- 
ification for the ADS analysis (see text). The samples used 
to produce the output are shown as histograms. The signal 
(Monte Carlo simulation) is the shaded histogram peaking 
near 1; the continuum (Monte Carlo simulation) is the his- 
togram peaking near 0. The off-peak data used to check the 
NN are overlaid as data points. 



B~ — ► D°K®tt~ production where the K®ir~ is nonreso- 
nant and has an invariant mass in the K*~ mass window. 
This background is discussed later. The second back- 
ground (peaking background) includes instances where a 
favored decay (e. g. B~ — > [K ~it + ]dK*~) contributes 
to fake candidates for the suppressed decay (i.e. B + — > 
[K~tt + ]]jK* + ). The most common way for this to occur 
is for a ir + from the rest of the event to be substituted 
for the 7T~ in the K*~ candidate. Other sources of peak- 
ing background include double particle-identification fail- 
ure in signal events that results in D° — ► K~ir + being 
reconstructed as D° — > tt~K + , or the kaon from the 
D being interchanged with the charged pion from the 
K* . We quantify this background with the ratio of the 
signal efficiency of wrong-sign decay to right-sign decay 
multiplied by the right-sign yield from data. The to- 
tal size of this right-sign pollution is estimated to be 
2.4 ± 0.3 events. Another class of backgrounds is charm- 
less decays with the same final state as the signal (e.g., 
B~ — > K*~ K + ir~). Since the branching fractions for 
many of these charmless decays have not been measured 
or are poorly measured, we use the D sideband to es- 
timate the contamination from this source. From a fit 
to the toes distribution using candidates in the D side- 
band we find 0.0 ± 1.1 events. We take the 1.1 events as 
the contribution to the systematic uncertainty from this 
source. 

Signal yields are determined from an unbinned ex- 
tended maximum likelihood fit to the Toes distribution 



in the range 5.2 < toes < 5.3 GeV/c 2 . A Gaussian func- 
tion (Q) is used to describe all signal shapes while the 
combinatorial background is modeled with an ARGUS 
threshold function (A) defined in Eq. (8). The mean and 
width of the Gaussian as well as the £ parameter of the 
ARGUS function are determined by the fit. For the like- 
lihood function we use a ■ A + b ■ Q where a is the number 
of background events and b the number of signal events. 
We correct b for the right-sign peaking background pre- 
viously discussed (2.4±0.3 events). 




5.2 5.22 5.24 5.26 5.28 5.3 
m ES (GeV/c 2 ) 



FIG. 4: Distributions of tties for the wrong-sign (top) and 
right-sign (bottom) decays. These decay categories are de- 
fined in the text. The dashed curve indicates the total back- 
ground contribution. It also includes the right-sign peak- 
ing background estimated from a Monte Carlo study for the 
wrong-sign (top) decays. The curves result from a simulta- 
neous fit to these distributions with identical PDFs for both 
samples. 

In Fig. 4 we show the results of a simultaneous fit to 
B- -> [K+n-] D K*- and B~ -> [K-^+] D K*~ can- 
didates that satisfy all selection criteria. It is in the 
wrong-sign decays that the interference we study takes 
place. Therefore in Fig. 5 we display the same fit sepa- 
rately for the wrong-sign decays of the B + and the B~ 
mesons. The results of the maximum likelihood fit are 
TLads = 0.066±0.031, Aads = -0.34±0.43, and 172.9± 
14.5 B~ — > [K~it + ]dK*~ right-sign events. Expressed 
in terms of the wrong-sign yield, the fit result is 11.5±5.3 
wrong-sign events (3.8 ± 3.4 B~ — > [K + it~]dK*~ and 
7.7 ± 4.2 B+ -> [K-tt+] d K*+ events). The uncertain- 
ties are statistical only. The correlation between IZads 
and Aads is insignificant. 

We summarize in Table IV the systematic uncertainties 
relevant to this analysis. Since both IZads and Aads 
are ratios of similar quantities, most potential sources of 
systematic uncertainties cancel. 
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TABLE IV: Summary of ADS systematic uncertainties. 



Source 



STZads 8 Aads 



5.28 5.3 

m ES (GeV/c 2 ) 



FIG. 5: The wrong-sign sample shown in the top plot of Fig. 4 
split by charge. Upper plot shows the tubs distribution of the 
B + — > [K~ tt + ]dK* + decays while lower plot presents the 
same for the B~ — * [K + -k~]dK*~ decays. The dashed curve 
indicates the total background contribution which includes 
the right-sign peaking background. 



For the estimation of the detection-efficiency asymme- 
try we use the previously mentioned results from the 
study carried out in Ref. [24]. We add linearly the cen- 
tral value and one-standard deviation in the most con- 
servative direction to assign a systematic uncertainty of 
SA c h — ±0.022 to the Aads measurement. To a good 
approximation the systematic uncertainty in IZads due 
to this source is STZads = TZads • Aads • 5A ch . 

To estimate the systematic uncertainty on Aads and 
TZads due to the peaking background, we use the statis- 
tical uncertainty on this quantity, ±0.3 events. With 
approximately 12 B~ — > [K + it~]dK*~ events and 
173 B~ — > [K ~it + ]dK *~ events this source contributes 
±0.002 and ±0.024 to the systematic uncertainties on 
TZads and Aads, respectively. Similarly, the 1.1 events 
uncertainty on the same-final-state background leads to 
systematic uncertainties of ±0.0061 and ±0.091 on TZads 
and Aads respectively. 

As in Section III, we need to estimate the systematic 
effect due to the non-resonant K^ir^ pairs in the K* 
mass range. We follow the same procedure discussed in 
Section III. After adding in quadrature the individual 
systematic uncertainty contributions, listed in Table IV, 
we find: 



Detection asymmetry 
Peaking bkg. 
Same-final-state bkg. 
Non resonant K®it~ bkg. 



±0.0005 ±0.022 
±0.0020 ±0.024 
±0.0061 ±0.091 
±0.0073 ±0.126 



Total systematic uncertainty ±0.0097 ±0.159 



V. COMBINED RESULTS 

We use the GLW and ADS results and a frequentist 
statistical approach [27] to extract information on 
and 7. In this technique, a y 2 is calculated using the 
differences between the measured and theoretical values 
and the statistical and systematic errors of the six mea- 
sured quantities. The values of td and 5d are taken from 
Ref. [12], while we allow < r B < 1, 0° < 7 < 180°, and 
0° < 5 B < 360°. The minimum of the \ 2 for the r Bl 
7, and 8b parameter space is calculated first (x m in)- We 
then scan the range of tb and 7 minimizing the \ 2 (Xm) 
by varying 5b ■ A confidence level for each value of rs and 
7 is calculated using A\ 2 = X m — Xmin and one degree 
of freedom. We assume Gaussian measurement uncer- 
tainties and confirm this assumption using simulations. 
In Fig. 6 we show the 95% confidence level contours for 
tb versus 7 as well as the 68% confidence level contours 
for the GLW and the combined GLW and ADS analysis 
(striped areas). 

In order to find confidence levels for tb we use the 
above procedure, minimizing \ 2 with respect to 7 and 
5b- The results of this calculation are shown in Fig. 7. 
The fit which uses both the ADS and GLW results has 
its minimum \ 2 at tb — 0.31 with a one sigma interval 
of [0.24, 0.38] and a two sigma interval of [0.17, 0.43]. 
The value tb = is excluded at the 3.3 sigma level. We 
find similar results for tb using the modifications to this 
frequentist approach discussed in Ref. [28] and using the 
Bayesian approach of Ref. [29] . 

Using the above procedure we also find confidence in- 
tervals for 7. The results of the scan in 7 are shown in 
Fig. 8. The combined GLW±ADS analysis excludes val- 
ues of 7 in the regions [0, 7]°, [62, 124]° and [175, 180]° 
at the one sigma level and [85, 99]° at the two sigma 
level. The use of the measurement of the strong phase 
5d [12] helps to resolve the ambiguities on 7 and there- 
fore explains the asymmetry in the confidence level plot 
shown in Fig. 8. 



VI. SUMMARY 



Aads — — 0-34 ± 0.43(stat.) ± 0.16(syst.) In summary, we present improved measurements of 

TZads = 0.066 ± 0.031(stat.) ± 0.010(syst.). yields from B~ DK*- decays, where the neutral 

D-meson decays into final states of even and odd CP 
(GLW), and the K + tt~ final state (ADS). We express 
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FIG. 6: 95% confidence level contours from a two dimen- 
sional scan of 7 versus tb from the B~ — > DK*~ GLW and 
ADS measurements. Also shown are the 68% confidence level 
regions (striped areas) for the GLW and the fit which uses 
both the GLW and ADS measurements, ro and 8d are from 
Ref. [12]. 



the results as IZcp, Acp, %±, 7£ ads, an d Aads- The 
value tb — is excluded at the 3.3 sigma level. These 
results in combination with other GLW, ADS, and Dalitz 
type analyses improve our knowledge of tb and 7. 
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FIG. 7: Constraints on r B from the B~ — > DK*~ GLW and 
ADS measurements. The dashed (dotted) curve shows 1 mi- 
nus the confidence level to exclude the abscissa value as a 
function of rg derived from the GLW (ADS) measurements. 
The GLW+ADS result (solid line and shaded area) is from 
a fit which uses both the GLW and ADS measurements as 
well as ro and 5d from [12]. The horizontal lines show the 
exclusion limits at the 1, and 2 standard deviation levels. 
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FIG. 8: Constraints on 7 from a fit which uses both the B~ — > 
DK*~ GLW and ADS measurements as well as td and 5d 
from [12]. The horizontal lines show the exclusion limits at 
the 1 and 2 standard deviation levels. 
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